feat: polymorphic helper for request body streaming by tank-bohr · Pull Request #477 · elixir-mint/mint

tank-bohr · 2026-03-19T08:59:49Z

Hey guys. Let me explain the problem I faced with. Below is the typical code for the sreaming body request

defmodule MintDialyzerRepro do
  @default_chunk_size 16_384

  def run(method, %URI{scheme: scheme} = uri, headers, body, opts)
      when scheme in ~w[http https] do
    scheme = String.to_existing_atom(scheme)
    port = uri.port || URI.default_port(uri.scheme)
    path = URI.to_string(%URI{uri | scheme: nil, host: nil, authority: nil})

    with {:ok, conn} <- Mint.HTTP.connect(scheme, uri.host, port, opts) do
      stream(conn, method, path, headers, body)
    end
  end

  defp stream(conn, method, path, headers, body) do
    with {:ok, conn, ref} <- Mint.HTTP.request(conn, method, path, headers, :stream) do
      stream_body(conn, ref, body)
    end
  end

  defp stream_body(conn, ref, "") do
    Mint.HTTP.stream_request_body(conn, ref, :eof)
  end

  defp stream_body(conn, ref, body) do
    chunk_size = min(chunk_size(conn, ref), byte_size(body))
    <<chunk::binary-size(chunk_size), rest::binary>> = body

    with {:ok, conn} <- Mint.HTTP.stream_request_body(conn, ref, chunk) do
      stream_body(conn, ref, rest)
    end
  end

  defp chunk_size(conn, ref) do
    case Mint.HTTP.protocol(conn) do
      :http1 ->
        @default_chunk_size

      :http2 ->
        chunk_size_http2(conn, ref)
    end
  end

  defp chunk_size_http2(conn, ref) do
    min(
      Mint.HTTP2.get_window_size(conn, :connection),
      Mint.HTTP2.get_window_size(conn, {:request, ref})
    )
  end
end

This code generates dialyzer warnings

Total errors: 2, Skipped: 0, Unnecessary Skips: 0
done in 0m1.78s
lib/mint_dialyzer_repro.ex:46:34:call_without_opaque
Function call without opaqueness type mismatch.

Call does not have expected opaque term of type Mint.HTTP2.t() in the 1st position.

Mint.HTTP2.get_window_size(_conn :: Mint.HTTP.t(), :connection)

________________________________________________________________________________
lib/mint_dialyzer_repro.ex:47:34:call_without_opaque
Function call without opaqueness type mismatch.

Call does not have expected opaque term of type Mint.HTTP2.t() in the 1st position.

Mint.HTTP2.get_window_size(_conn :: Mint.HTTP.t(), {:request, reference()})

________________________________________________________________________________
done (warnings were emitted)
Halting VM with exit status 2

It was discussed in the #380 and was left with a dirty workaround.
And people are using the workaround. And on the OTP-28 such trick in the k8s library leads to the infinite hanging during the PLT building phase.

But it's a legit issue. And it's totally worth to be fixed.

Opaqueness

Since the caller have to know the exact type for streaming, the Mint.HTTP.t() cannot be opaque. And the caller cannot know the exact type in advance either, because it's not their decision, but the server's decision.

Missing API for streaming

Another consequence of this code and dialyzer warning is the request body streaming API is not sufficient for using it without knowing the underlying protocol. But it's kinda designed to be such. To close the gap the current PR is introducing the missing piece - an ability to get the chunk size for the streaming.

tank-bohr · 2026-04-22T09:41:32Z

@ericmj Could you please take a look

whatyouhide · 2026-04-23T16:35:55Z

What if you make

@type Mint.HTTP.t() :: Mint.HTTP1.t() | Mint.HTTP2.t()

so that the "top-level" type is not opaque, but the inner types are?

tank-bohr · 2026-04-23T18:01:57Z

What if you make
@type Mint.HTTP.t() :: Mint.HTTP1.t() | Mint.HTTP2.t()
so that the "top-level" type is not opaque, but the inner types are?

@whatyouhide That's exactly what my PR does. It un-opaques (exposes) the Mint.HTTP.t(). This will help to solve the dialyzer issue in the existing code (like k8s). Because existing code has to know about internals.

And get_send_window callback and Mint.HTTP.next_body_chunk_size/3 potentially can allow to opaque the Mint.HTTP.t() type as well. But it requires changing the callers code. But it's helpful anyhow. It brings the missing piece to the ubiquities API of Mint.HTTP

tank-bohr · 2026-04-23T18:11:28Z

the original example with the get_send_window will look like

defmodule MintDialyzerRepro do
  def run(method, %URI{scheme: scheme} = uri, headers, body, opts)
      when scheme in ~w[http https] do
    scheme = String.to_existing_atom(scheme)
    port = uri.port || URI.default_port(uri.scheme)
    path = URI.to_string(%URI{uri | scheme: nil, host: nil, authority: nil})

    with {:ok, conn} <- Mint.HTTP.connect(scheme, uri.host, port, opts) do
      stream(conn, method, path, headers, body)
    end
  end

  defp stream(conn, method, path, headers, body) do
    with {:ok, conn, ref} <- Mint.HTTP.request(conn, method, path, headers, :stream) do
      stream_body(conn, ref, body)
    end
  end

  defp stream_body(conn, ref, "") do
    Mint.HTTP.stream_request_body(conn, ref, :eof)
  end

  defp stream_body(conn, ref, body) do
    chunk_size = Mint.HTTP.next_body_chunk_size(conn, ref, body)
    <<chunk::binary-size(chunk_size), rest::binary>> = body

    with {:ok, conn} <- Mint.HTTP.stream_request_body(conn, ref, chunk) do
      stream_body(conn, ref, rest)
    end
  end
end

As you see no need for using the HTTP2 module directly. No need to detect protocol. The Mint.HTTP module dispatches to the correct implementation itself. With this change we can keep Mint.HTTP.t() opaque. But I still recommend to unopaque it because of the existing code

ericmj · 2026-04-23T21:56:42Z

  end

+  @impl true
+  def get_send_window(%__MODULE__{streaming_request: %{ref: ref}, sndbuf: sndbuf}, ref),


This is semantically very different to what what the HTTP2 module does. There is no concept of send windows in HTTP/1 so we should not expose it on this module.

ericmj · 2026-04-23T21:57:36Z

  @behaviour Mint.Core.Conn

-  @opaque t() :: Mint.HTTP1.t() | Mint.HTTP2.t()
+  @type t() :: Mint.HTTP1.t() | Mint.HTTP2.t()


Suggested change

@type t() :: Mint.HTTP1.t() | Mint.HTTP2.t()

@type t() :: Mint.HTTP1.t() | Mint.HTTP2.t() | Mint.UnsafeProxy.t()

ericmj · 2026-04-23T21:59:42Z

  end

+  @impl true
+  def get_send_window(%__MODULE__{} = conn, ref),


get_window_size/2 is already public, I don't think we need a function that only calls min on top of that.

ericmj · 2026-04-23T22:11:23Z

+  def get_send_window(conn, ref), do: conn_apply(conn, :get_send_window, [conn, ref])
+
+  @spec next_body_chunk_size(t(), term(), binary()) :: non_neg_integer()
+  def next_body_chunk_size(conn, ref, body), do: min(get_send_window(conn, ref), byte_size(body))


I think this function is unnecessary.

feat: polymorphic helper for request body streaming

015c79b

ericmj reviewed Apr 23, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: polymorphic helper for request body streaming#477

feat: polymorphic helper for request body streaming#477
tank-bohr wants to merge 1 commit intoelixir-mint:mainfrom
tank-bohr:main

tank-bohr commented Mar 19, 2026 •

edited

Loading

Uh oh!

tank-bohr commented Apr 22, 2026

Uh oh!

whatyouhide commented Apr 23, 2026

Uh oh!

tank-bohr commented Apr 23, 2026 •

edited

Loading

Uh oh!

tank-bohr commented Apr 23, 2026 •

edited

Loading

Uh oh!

ericmj Apr 23, 2026

Uh oh!

ericmj Apr 23, 2026

Uh oh!

ericmj Apr 23, 2026

Uh oh!

ericmj Apr 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	@type t() :: Mint.HTTP1.t() \| Mint.HTTP2.t()
	@type t() :: Mint.HTTP1.t() \| Mint.HTTP2.t() \| Mint.UnsafeProxy.t()

Conversation

tank-bohr commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Opaqueness

Missing API for streaming

Uh oh!

tank-bohr commented Apr 22, 2026

Uh oh!

whatyouhide commented Apr 23, 2026

Uh oh!

tank-bohr commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tank-bohr commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ericmj Apr 23, 2026

Choose a reason for hiding this comment

Uh oh!

ericmj Apr 23, 2026

Choose a reason for hiding this comment

Uh oh!

ericmj Apr 23, 2026

Choose a reason for hiding this comment

Uh oh!

ericmj Apr 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tank-bohr commented Mar 19, 2026 •

edited

Loading

tank-bohr commented Apr 23, 2026 •

edited

Loading

tank-bohr commented Apr 23, 2026 •

edited

Loading